Robust Distributed Optimization With Randomly Corrupted Gradients

In this paper, we propose a first-order distributed optimization algorithm that is provably robust to Byzantine failures-arbitrary and potentially adversarial behavior, where all the participating agents are prone failure. We model each agent's state over time as two-state Markov chain indicates or trustworthy behaviors at different instants. set no restrictions on maximum number of any given time. design our method based three layers defense: 1) temporal aggregation, 2) spatial 3) gradient normalization. study two settings for stochastic optimization, namely Sample Average Approximation Stochastic Approximation. provide convergence guarantees strongly convex smooth non-convex cost functions.